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^ ' Missing link prediction in indirected and un- weighted network is an open and challenge problem which has 

been studied intensively in recent years. In this paper, we studied the relationships between community 
structure and link formation and proposed a Fast Block probabilistic Modcl(FBM). In accordance with the 
experiments on four real world networks, we have yielded very good accuracy of missing link prediction 
and huge improvement in computing efficiency compared to conventional methods. By analyzing the 
mechanism of link formation, we also discovered that clique structure plays a significant role to help us 
• understand how links grow in communities. Therefore, we summarized three principles which are proved 

to be able to well explain the mechanism of link formation and network evolution from the theory of 
Q 1 graph topology. 
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Recently, the study of link prediction has attracted much attention from disparate scientific communities. 
In the theoretical aspect, accurate prediction indeed gives evidence to some underlying mechanisms that 
drive the network evolution pQ. Moveover, it is very possible to build a fair evaluation platform for network 
modeling under the framework of link prediction, which might be interested by network scientists [2][3] . 
In the practical aspect, for biological networks such as protein-protein interaction networks and metabolic 
networks 4 6 , the experiments of uncovering new links or interactions are costly, and thus to predict 
in advance and focus on the links most likely to exist can sharply reduce experimental costs [TJ. In 
the Facebook social network, the link prediction based on supervised random walks has been applied 
to predict and recommend possible future friends to the current user |S]. link prediction approach is 
also verified very useful for human preferences recommendation in the field of so called collaborative 
filtering [9]. In Massively Multiplaycr Online Role Playing Gamc(MMORPG) networks, the studies of 
link prediction have given an insight to uncover underlying relationships between game players |10H12j . 
In recent years, there are various research fields developed by the studies of link prediction in complex 
networks. Missing link prediction, as a fundamental issue, aims at estimating the likelihood of the 
existence of a link between two nodes in a given network based on the observed links [2j[13l[T4]. Similar 
to the missing link prediction, spurious link prediction is also carefully studied by some researchers |15) . 
In online social networks, determining positive and negative links is another interesting problem of link 
prediction which has aroused people's attention recently [TB] because researchers believe that relations 
such as friendship should be opposite to other relations such as antagonism. On the other hand, since 
network is very dynamic, given link data for times 1 through T, can we predict the links at time T+l? 
Therefore, researchers also try to investigate the issue of temporal link prediction [17]. Likewise, the 
link prediction in multiple networks become possible as well, which means the task to predict links in a 
network by only using features from other networks |10j . 
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As for the studying fields mentioned above, various models and methods have been proposed. Some 
conventional algorithms mainly take the link prediction as a classification issue and therefore apply 
classical machine learning methods based on node attributes information [18H21] . and the attributes, for 
example, would be a people's age, sex, friend prefence, and so on in a social network or a computer's 
IP adress, MAC address, operating system type, etc in a computer network. However, scientists have 
found that network's structure information is more powerful than nodes attributes in most contexts and 
proposed many simple but effective methods merely utilizing local or global structure information of the 
network, such as the methods of common neighbor, Jaccard, Katz [22], Adamic Adar [23) . and resource 
allocation [53], etc. Recently, there are several hybrid methods that attemt to combine node attributes 
with structure information appeared in some literatures as well [8][10], but most of them are applied in 
some specified domains which mean that domain knowledge must be required. 

Since link prediction based on structure features of the network is more general, in this paper, we focus 
on the issue of missing link prediction in indirected and un-weightcd networks by merely using network's 
topology information. As we believe the community would be the cradle to promatc link formation, we 
present a novel model based on the community structure and statisitc theory. To assess the performance 
of our model, we evaluate it on four real world networks. Results show that our model improves both 
the prediction accuracy and computing efficiency. Due to the vital theoretical interest to understand the 
mechanism of link formation, we evolve a encouraging theory framework to explain this issue which is 
proved sound by our experiments. 



Community model of the network 

Communities, which are also called modules or clusters, exist widely in real world networks. Intuitively, 
a community could be a group of nodes with dense connections within a network. Since links tend 
to crowd in the community, this enlightens us to explore if there exists any underlying correlations 
between community evolution and link formation and this is also the motivation for us to pursue a 
new link prediction method based on the distribution of the community structure in a given network. 
Therefore, We need to analysis the community structure of the network and study the approach to find 
the communities properly. 

Community detection is a fundamental task to exploit the blocks or subgraphs with different properties 
and functions nested in a network. For social networks, a community could be a group of people with 
common interest or location. For biology networks, a community could be a group of cells or proteins with 
common function. Current community detection approaches are "biased" for they are often related to 
some complex structural features such as sparsity, heavy-tailed degree distribution, and short diameter, 
etc and also strongly depend on the specific application [25H27] , As a result, so far, there is no such a 
universal measure which is able to determine whether a community obtained by any of these approaches 
in a given network is true or not fairly. In this paper, we use the measure of link density to quantitively 
ascertain whether a block is a community or not. If a block [i has nodes with number |V^| and edges 
with number the link density of the block is defined as follows, 



Where m M = 1-6^1 and = V M |(|V^| — l)/2. According to Eq. (fT]), The link density denotes the 
ratio of actual number of inner links to maximal possible number of links in the block \x. Notice that, 
when the equals 1, the block will reach the highest density and form a complete subgraph which 
is also called a clique. To quantitively describe the link density between two blocks, we also define the 
connecting density between every two blocks fi and v in a given network as follows. Supposing \E^ V \ 
is the number of edges between two blocks while |V^| and \V V \ are the number of nodes in the block \i 
and block v respectively. | | multiplying | V v | denotes the maximal number of links possiblely existed 
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between the two blocks. 
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Where = \Ey, v \ and n^ v = |V^||V^,|. Here, we define a community as such a block in a given 

network which has relatively high inner link density and relatively low connecting density with other 
blocks. We notice that the structure and characteristic of the communities have naturally exhibited a 
statistic mechanism of which links are more likely to emerge in the community whereas are less likely to 
be established between communities. If a network could be partitioned into communities properly, we 
will be likely to estimate the probability of any node pairs in terms of the distribution of communities. 
In Fig. Ufa) , we give an example of network which has been partitioned into three possible communities 
based on our community definition which are marked with different colors. The conventional community 
detection methods tend to bind every node in the network to some particular communities enforcedly. 
But, by doing so, some leaf nodes will be introduced into the community as noise to decreases the inner 
link density of the community, and this is so-called issue of resolution limit of community detection |28j . 
And we think this is particularly true in scale-free networks since there are enormous leaf nodes in 
such kind of network. Unlike the conventional point of view for community detection, we consider that 
these nodes may not belong to any communities and should be categorized as a group of isolated nodes. 
Therefore, those leaf nodes which are marked with brown color in Fig. HJa) are grouped together as a 
special "community" which has no inner links. Of course, in this "community", links have very little 
chance to be established among node pairs. If we partition a network into communities in this manner, we 
can obtain a link density distribution matrix by using Eq. (p} and Eq. ([2]) to calculate the density within 
and between the communities. As for Fig. [Ha), we have partitioned the network into four communities 
including a special " community" , thereafter we yield a link density matrix shown in Fig. [Ub) . 
One network partition can only provide one link density distribution while there usually exist various 
possible network partitions. If we want to estimate the connecting probabilities for all node pairs in 
a given network, based on the theory of statistics, we need to obtain independent network partitions 
as many as possible by doing multiple rounds of network partition. Such procedure is also known as 
sampling. Considering an observed network with a fraction of links removed has adjacency matrix A° , 
we apply a block patition model B to the observed network. According to the Bayes theorem, the link 
probability of a node pair Xij can be estimated as 



where denotes the space of sampling. For a node pair in an obtained block [i by model B, we 
suppose p(xij\A° , bfj) = pij. Due to having the high link density in the block fi, it meets that m M — > n^. 
We have that 



p{ XlJ \A°) 



J2n J B pjxij \A°, B)p(A°\B)p(B)dB 
J2 n J B p(AO\B)p(B)dB 



(3) 
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Using Eq. ^ and Eq. ((5J), one can obtain 
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Likewise, for a node pair Xij, where node i is in block \i and node j is in block v, we also suppose 
p(xij\A°, b^y) = pij. Due to the low connecting density between the block p, and block v, it meets that 
TUfiv <C ti^tj. We have that 

P (A°\b flv )^p^(i- Pij r^. (?) 

Let p{b^ v ) equal a constant. Eq. ((3]) can be rewritten as 

, lAO) En/oP(*ijl^M^°IM'fr 
EaIoP^°Kv)dp 



Using Eq. ([7} and Eq. ([5]), one can obtain 

p(*iM°) = t : t^t- 

/ n^ v + ra^v + 1 



n (m Mt) + 2) 



EnK« + 1 

In the experiments and evaluations section, we mainly use Eq. ([6]) and Eq. ([9]) to estimate link proba- 
bilities for all node pairs which have no links in the observed network. 



Fast block probabilistic model 

According to the network community model in the previous section, our goal is to partition the network 
into a set of blocks and ensure each block either is a community or a special community, namely a group 
of isolated nodes having no inner links. It's not trivial to find out all communities in a given network 
by exhaustive searching because the searching space is usually over large. To fulfill the above task, we 
proposed a Fast Block probabilistic Modcl(FBM) by using greedy strategy. Comparing to conventional 
methods using the rule of Metropolis-Hasting |29l , our algorithm has obtained huge improvment in com- 
puting efficiency in accordance with the pcformance evaluation results shown in the next section. The 
FBM Alogorithm is formulated in Table Q] and Table [2] 

We don't need to provide a mechanism in our algorithm to ensure the relatively low connecting density be- 
tween blocks because of the fact that almost all real- world networks are sparse network. By implementing 
our algorithm, we find that it can keep the rare links between obtained every two blocks automatically, 
i.e., every two blocks has relatively low connecting density. Actually, We have also verified that our 
algorithm can still work well even through the network is dense. Based on the algorithm, we can quickly 
partition a given network into communities with high link density and two special communities which 
are merely grouped by isolated nodes due to the first step in our algorithm of which wc initially partition 
the network into two blocks randomly. Please note this step is essential in our algorithm. Without this 
step, the network partitions obtained will be strongly correlated. To ensure all network partitions are in- 
dependent to each other, randomly partition the network into two blocks before implimenting the greedy 
search of communities is a simple but effective trick in our algorithm which can trigger the procedure 
of sampling(thc independent relationship can be validated by mutual information between partitions). 
On the other hand, if we remove this step, our algorithm will transform to a pure community detection 
algorithm. Despite the theoretical and practical interests of community detection, we will not give further 
investigations on this issue but keep focus on the study of miss link prediction. 



Experiments and evaluations 

In this paper, we consider four real-world networks to implement the tests and evaluations. (1) Social 
network of friendships between 34 members of a karate club at a US university in the 1970s [30]. (2) 
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Food web of a grassland ecosystem, i.e., a network of predator-prey interactions between species |31j.(3) A 
network of associations between terrorists [35]. (4)C. clcgans (CE): The neural network of the nematode 
worm C. elcgans, in which an edge joins two neurons if they are connected by either a synapse or a gap 
junction |33j . Here, we only consider the giant component and every network is treated as an indirectcd 
network. The topological features statistics of the four networks are summarized in Table [3] 
Before implementing comparisons with other link prediction methods, we still need to determine an un- 
certain parameter in our alogrithm. As stated in the Table [21 the threshold of the link density must be 
chosen carefully. We try to pick out the optimized value of the threshold of link density by observing 
the variation tendency of prediction accuracy along with different link density settings. The measure for 
prediction accuracy we used here is AUC(area under the receiver operating characteristic curve). AUC 
can be interpreted as the probability that a randomly chosen missing link is given a higher score than 
a randomly chosen non-existent link. Fig. [2] shows the accuracy variation curves ploted for the four 
networks as the fraction of missing links is set to ten percent- We found the accuracy of link prediction 
tends to coverage after the link density is larger than 0.5 and reaches the best when the threshold of link 
density is set to 1 while the block corresponds to a clique. 

To estimate the likelihood of missing links, researchers have developed various probabilistic prediction 
models in recent years. A typical model, Hierarchical Random Graph(HRG), proposed by Aaron Clausct 
et al, was applied to predict miss links in some networks with obvious hierarchical structure [8]. Based 
on the similar theory of statistics using by HRG model but from another angle of view, Roger Guimera 
et al proposed a Stochastic Block Modcl(SBM) which can predict both missing links and spurious links 
and is able to give a much better accuracy of prediction in various kind of networks compared to some 
other popular methods including the HRG [10] approach. To our best knowledge, the SBM algorithm is 
the state-of-the-art approach with the best accuracy of prediction in indirected network without weight 
information. 

We mainly made performance comparisons both on missing link prediction accuracy and computing ef- 
ficiency between our algorithm and the SBM approach. And the accuracies of the common neighbor 
method are presented here as a baseline. Our algorithm and the SBM approach have a common char- 
acteristic which are both required to sample network partitions. To ensure the comparison is fair to the 
both approaches, we apply the same sampling standard to them which is set to 50 times. The hardware 
we use to test is a desktop with processor of Intcl(R) Core(TM) i7 CPU 930 @ 2.8Ghz(eight cores) and 
8 GigaBytes memory. 

The prediction accuracies, measured by AUC for the four networks, are plotted in Fig. [3jand the corre- 
sponding comparisons of running time(the unit is second) implimcntcd in the four networks are shown 
in Fig. [4] To ensure the results arc trusted, each value of accuracy is obtained by averaging over 100 
implementations with independently random network divisions of training set and probe set while the 
error bars denote the standard deviation. Accordingly, each value of the running time is the lasting 
period over 100 implementations. 

According to the AUC comparisons shown in Fig. [3] the FBM approach performs better than the SBM 
approach in the networks of Grassweb and Terrorist, and has very close accuracy result to that of the 
SBM approach in the other two networks. Meanwhile, According to the computing efficiency comparisons 
shown in Fig. 2] the running time used by the FBM approach are far less than that used by the SBM 
approach. During the experiments, we found that the running time consumed by the SBM approach 
increases rapidly along with the size growth of the network while that of the FBM approach increases 
mildly, and this indicates that the SBM algothim has much high time complexity than the FBM algorithm 
and is also the main reason that why we have chosen the networks with relatively small size to make the 
comparisons. The experiment results prove that the FBM approach is able to give very good accuracy 
for missing link prediction on real world networks with superior computing efficiency. 



'we obtained similar results as other different fractions of missing links are set to. 
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Mechanism analysis of the link formation 

To find out the reason that why the FBM approach is of the capacity to give very good accuracy for 
missing link prediction in real world networks, we try to analyze what kind of links have higher link 
probability by investigating the probabilistic distributions of node pairs in the four networks we have 
tested. We observed there are mainly three important principles which may build the theoretical basis 
to explain what kinds of links are likely to emerge in the networks. To demonstrate the three principles 
easily, we give three example networks shown in Fig. [SJ each of which is likely to uncover one kind of 
links favored by the FBM model. 

Fig. Eta) shows a ring structure with six nodes labeled by numbers and two possible links between node 
pair (1,3) and node pair (3,6) which arc denoted by red dash line and blue dash line. We evaluate the 
likelihoods of the two possible connections by applying the FBM approach to the network. The results 
show that the connection probability of node pair (3,6) is only 50 percent of that of node pair (1,3) which 
means that the node pair (1,3) is more likely to connect together compared to node pair (3,6). We notice 
that, if a link added between node pair (1,3), a clique (1,2,3) would be established. This indicates that 
link tends to in priority establish a clique in a network. Fig. [5jb) shows a five-node network and two 
possible links between node pair (1,3) and node pair (3,5) which are also denoted by blue dash line and 
red dash line respectively. After calculating the link probabilities of the two node pairs, we found that 
the connection probability of node pair (1,3) is 75 percent of that of node pair (3,5). The difference 
of the two connection probabilities reminds us that an addition of link (3,5) will form a larger clique 
(2,3,4,5) than the clique (1,2,3) if link (1,3) added. This case demonstrates that link tends to create 
larger clique first in a network when there are many options available to choose. Fig. [He) shows another 
interesting phenomenon of link formation. After calculation, the link probabilities of node pair (3,5) is 
1.5 times higher than that of node pair (1,3). We found that link (3,5) could create three cliques including 
(1,2, 5), (2, 3, 5) and (2,4,5) while link (1,3) would only be able to create two cliques, i.e., (1,2,3) and (1,3,5). 
This result implies that if adding a link is able to create more cliques, the link will have higher likelihood 
to be established. In terms of the three typical cases, we summarized three principles to explain how link 
creates. 

(i) Link is very likely to be established to form a clique in a given network. 

(ii) Link prefers to create larger clique to smaller clique in a given network, 
(hi) Link tends to form cliques as many as possible in a given network. 

As stated in section 2, links tend to crowd in the communities. We believe the three principles would be 
reasonable to cxplian the link formation mechanism in the communities. To prove this hypothesis, we 
made an addtional cxpriment on the four networks which have been used to test in the prior section. We 
firstly set the link density threshold to 0.8 and apply the FBM approach to partition each network into 
communities respectively. Then we remove 10 percent of links existed in the communities by following 
the three princples as probe set and keep the rest of the network as training set. We still use the AUC 
to evaluate the accuracy of prediction and obtain the average results upon 100 time implementations 
shown in Table [4] Comparing to the results shown in the prior section, we found that the new predicition 
results are even better which prove that the FBM has definitely applied the three principles to predict 
the missing links within the communities. In other word, it has also proved that the three principles can 
well capture the essence of link formation and reveal the rule of community growing and evoluation in 
real world networks. 

The mechanism of common neighbor approach is usually explained by the social balance theory |34U35j . 
but it is actually a special case which has applied the principle (iii), since if a given node pair has many 
common neighbors, it also means that it would form many triangle cliques after a link is added to the 
node pair. Therefore, this approach can still perform well in some cases. But due to only partial essense 
of link formation captured by this method, it could not give good accuracy results in our experiments. 
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Discussion 

The three principles have suggested that the clique stucture plays a significant role to drive link formation 
in the communites. In this section, we want to give more empirical evidences provided by other researcher 
to discuss why clique can fulfil this task. Compared to other structures, such as asterial structure and 
ring structure, the clique has some unique features. Firstly, the clique is the most complete structure 
which has the most dense links and hence ensures it is the most stable and robust component of the 
network. Secondly, the clique has shortest distance between every two nodes which ensure it has very 
high efficiency for communication. It has been revealed that, in Internet network, some hub nodes like 
backbone routers tend to connect together which is so-called rich-club phenomenon [3(Jj- The rich-club 
will improve the efficiency of traffic routing and provide the capacity to resist some node attacks and 
prevent the network from breaking down easily. As rich-club is a typical community in terms of our 
definition, the clique might have the underlying impact to promote the rich-club's formation and growth 
in the network. Its also found in biology network that motifs often have the structure of the clique, such as 
the feed-forward loop, which is known as directed triangle motif, emerges in both transcription-regulatory 
and neural networks [37] . Meanwhile, the research finding has revealed that these motifs tend to cluster 
together which exhibit as a general property in all real networks, so, new links tend to emerge during this 
clustering procedure. The above known research achievements in different domains have provided more 
solid empirical evidences to support our theory. 

Previous studies have also verified that the local clustering property such as nodes clustering coefficient 
can be utilized to improve the accuracy of link prediction, yet they did not give any solid reason about 
their methods[I9]. Clique is a typical stucture which is of the property of local clustering in the network. 
Our work has revealed that such structure is the important cradle to promote link formation in community 
and network evolution. With an intuitive insight, we also believe that our model has the potential to 
provide some new evidence to explain why so many networks in the real world exhibit the topology of 
hierarchy communities if further network evolution study can be done under the link formation mechanism 
we've found. 

Conclusions 

In this paper, we proposed a Fast probability Block Model(FBM) to predict missing links in complex 
networks. In terms of the experiment results in four real-world networks, the FBM model has exhibited 
slight better accuracy performance and overwhelmed better computing efficiency than the state-of-the- 
art model SBM. We believe the FBM approach has the potential to give fairly good accuracy of link 
prediction in much larger and more complex networks such as massive biology networks, rapid growing 
social networks, and World Wide Web networks, etc. So, compared to the SBM approach, The FBM 
algorithm is more applicable. On the other hand, from the theoretical aspect, we revealed that networks 
clique structure plays an important role to drive link formation and community evolution. And the 
underlying mechanism of link formation in communities have been well interpreted by three principles 
summarized in this paper which can provide researchers a new framework for link prediction study. 
Meanwhile, our model is very likely to give enhanced prediction accuracy in specific applications when 
domain knowledge is introduced such as node properties and edge features, furthermore, FBM model is 
of a good outlook to give an insight on exploring new methodology of community detection in complex 
networks although this is not investigated deeply in this paper. 
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(a) (b) 



Figure 1. An example network to interpret the relationship between community distribution and link 
density matrix, (a) The community distribution of the network, (b) The link density matrix of the 
community distribution 




Figure 2. Correlations between AUCs and thresholds of link density in the four networks 
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Figure 3. Accuracy comparison of link prediction approaches in four networks. Each AUC is avergaed 
upon 100 implementations and error bar represents the standard deviation. 



Table 1. The algorithm of Fast Block probabilistic Model 

Input: a network G(V, E); 

Output: communities (Gi, C 2l C m ) of the network G; 

1. Let Gi = (V^E-l) and G 2 = (V 2 ,E 2 ) where V = V x U V 2 , <P = V\ n V 2 and <P = E x n E 2 ; 
/*the blocks G\ and G 2 are partitioned randomly by the network G(V, E)*/ 

2. For each Gi do 

3. j=l] 

4. While Ei <> <j> do /* loop stops when no edges exist in Gi*/ 

5. CommunityFind(Gi,Cj); /*procedure to find a community Cj from Gi with high density*/ 

6. Output(Cj); 

7. G;=Rcmove(Cj, Gi); /*remove the community Cj from Gi including nodes and 

edges which belong to Cj and interconnections between Cj and Gi*/ 

8. j=j+l; 

9. End while 

10. Output(Gj); /*the remaining graph Gi will be a group of isolated nodes*/ 

11. End for 
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Figure 4. Computing efficiency comparison in four networks. Each running time is the lasting period 
of 100 implementations. 






Figure 5. Three artificial networks to interpret the link formation mechanism in the communities 



13 



Table 2. The procedure of CommunityFind called by the algorithm of Fast Block 
probabilistic Model 

Input: a network Gf, 
Output: a community Cf, 

1. ^4=Density(Gi); /*calculatc the density of the network G%*/ 

2. while A < threshold do /^threshold is an accepted maximum value of link density */ 

3. Sort(Vi); /*sort all nodes by node degree in descending order*/ 

4. Gj=Remove(w,Gi); /*remove the node v in the top of the list with the least degree and edges 
attached to the v from Gi, and derive a new network Gi* / 

5. A=Density(Gj); /*recalculate the density of network Gi* / 

6. End while 

7. Cj=Gi\ 

8. Return(Cj); /*derive a community Cj with threshold density */ 



Table 3. Topological features statistics of the four networks. \V\ and \E\ are the number of 
nodes and links. C and D are clustering coefficient and density of network, respectively. 
M is the modularity of network, (k) and (d) are the average degree and the average 
shortest distance. 





\v\ 


\E\ 


C 


D 


M 


(k) 


(d) 


Karate 


34 


78 


0.588 


0.139 


0.416 


4.588 


2.408 


Foodweb 


75 


113 


0.497 


0.041 


0.635 


3.013 


3.875 


Terrorists 


62 


152 


0.58 


0.08 


0.529 


4.903 


2.508 


CE 


297 


2148 


0.308 


0.049 


0.397 


14.465 


2.946 



Table 4. The accuracy results of missing link predcition in communitis of the four 
networks. The threshold of link density to partition communites is set to 0.8 for each 
network. And the fraction of links removed from the communities is 10 percent. 



Network 


Karate 


Foodweb 


Terrorists 


CE 


AUC 


0.9427±0.0583 


0.9013±0.0471 


0.9585±0.0387 


0.9535±0.0076 



